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Preface 


Residents of Ontario communities are becoming increasingly aware of 
the impact of both small and large scale planning activities upon their 
daily lives. Whether these activities relate to the planning of a neigh- 
bourhood park or local traffic control, the establishment of a regional 
conservation area or a provincial transportation corridor, increased 
concern is being expressed about their various impacts upon the nature 
and quality of community life. As a result, agencies with public plan- 
ning responsibilities at the federal, provincial and municipal level are 
taking a more active role and are developing additional opportunities 
to involve a wider spectrum of the public in the planning process. 


The following discussion paper was developed from a concern that many 
of these initiatives have been developed and implemented without a 
clearly defined conceptual framework of the public participation pro- 
cess and have lacked a model of analysis for evaluation. To further 
the dialogue, the following model was developed by T. K. Eger and 
J. T. Johnston and was published in 1974. In response to demand 
and in order to incorporate revisions, this paper has been reprinted. 


—T Es 


T.K. Eger 

Community Development Unit 
Multicultural Development Branch 
1976 
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Introduction 


One of the major difficulties in developing evaluation tools for public participation 
programmes is the complex and subjective nature of the process itself. Notwith- 
standing the difficulties involved, there exists a need for documented empirical 
evidence as to the relative success of applying various participatory techniques and 
resource alternatives. Unfortunately, little attention has been given to the develop- 
ment of evaluative mechanisms, perhaps partly due to fear of negative evaluatory 
results or perhaps mainly due to the lack of any framework for analysis. What 
follows in this paper is an outline of the evaluation process, and a suggested model 
which might be applied or adapted. 


Outline of the Evaluation Process 


Evaluation Adapted from Tripodi, Fellin, and Epstein, (1971), evaluation is 

defined defined as the systematic gathering of data to provide information 
about the achievement of public participation programme requisites 
and goals relative to EFFORT, EFFECTIVENESS AND EFFICIENCY. 
With this perspective in mind, comprehensive evaluation of public 
participation programmes would include an analysis of: 


e the total input (EFFORT) 
e the total output (EFFECTIVENESS) and 


e the cost benefit ratio (EFFICIENCY) 


Approach While there appears to be a number of possible valid evaluation 
Outlined methods, the following three are identified as being common to many 
research activities: 


e an approach based upon the relative 
achievements of concrete definable 
objectives (a goal approach) 


e an approach based upon the extensive 
use of indicators (a systems approach) 


e an approach based upon the perception 
of the participants in the programme 
(a perceptions approach) 


There are obvious deficiencies in each approach applied singly. 
However, a combination of the above three is generally acknowledged 
to have advantages not fully realized when each is pursued on a sep- 
arate or parallel basis. 


Objectives An objective can be considered as something toward which effort is 

Described directed, an aim or an end of action. In order to evaluate objectives, 
they must be sufficiently specific and concrete to be measured. Ob- 
jectives must be set prior to determining what indicators are required 
for evaluation. 


Indicators One of the approaches which may be used to determine a comprehen- 

Described sive evaluative tool is the extensive use of indicators. Indicators are 
descriptive devices derived from social science research which en- 
able classification, measurement, and analysis of behaviour, results 
and processes. By developing social indicators researchers attempt 
to make concrete, identifiable and visible that which in the field of 
social science is often clouded by abstract concepts and theory. Social 
indicators then, are simply tools which the social science theorist or 
practitioner develops in order to understand and communicate more 
fully the dynamics of a situation. 


Participants Turning to the question of who should be involved in the evaluation 
Outlined process, the following key individuals or groups may be identified: 


e the participants or service users 
directly affected 


e selected or interested groups acting 
as representatives of the community 
at large 


e the administrators monitoring the 
programme 


e the elected representatives responsi- 
ble for final policy determination. 


An Overview 


Having listed three different approaches to evaluation, it is now 
necessary to outline an overview of the evaluation model. As stated 
previously, evaluation includes the systematic gathering of data to 
provide information about the achievement of public participation 
programme requisites and goals relative to EFFORT, EFFECTIVE- 
NESS and EFFICIENCY. Schematically, the evaluation process 
described to this point is illustrated in Fig. I. The Evaluation 


Approach. 


EFFORT 
Objectives 
Evaluation 
Indicators EFFECTIVENESS 
Approach 
Participants 
EFFICIENCY 


Fig. I. The Evaluation Approach 


PROGRAMME EFFORT 


This concept can be viewed as the type and quantity of activities 
considered necessary - regardless of output. If refers not only 
to staff time, activity, and commitment but also to the allocation 
and use of material resources - funds, space, equipment. 


The evaluator should be able to analyze public participation pro- 
gramme EFFORT by the use of Process and Performance Indica- 
tors. Further explanation of indicators is discussed later in this 
section. 


EFFORT is necessary for the achievement of programme goals. 
However, the quantitative, descriptive information of programme 
EFFORT indicates only how much staff and programme activity 

is being carried out. This does not indicate how well the tasks are 
being accomplished, or whether the programme's overall goals are 


being attained. Obviously, if there is little EFFORT invested ina 
programme, little will be accomplished. The evidence of programme 
activity alone is not sufficient to determine whether or not programme 
goals have been reached. For that analysis, the concept of programme 
EFFECTIVENESS needs to be examined. 


PROGRAMME EFFECTIVENESS 


This concept refers to the extent to which the goals and objectives of 
the public participation programme have been achieved. It is an exa- 
mination of the results or output of the programme with respect to the 
original goal. It is also an examination of the EFFORT with respect 

to what has been accomplished. An evaluation of EFFECTIVENESS 
would consider both desirable and undesirable unanticipated conse- 
quences of the programme EFFORT. It would also examine how well 
the tasks of the programme are being accomplished. Public participa= 
tion programme EFFECTIVENESS should be able to be analyzed by the 
use of effectiveness indicators and change indicators. 


PROGRAMME EFFICIENCY 


This concept refers to the ratio of EFFECTIVENESS to EFFORT in 
terms of such costs as money, time, personnel, convenience, and 
physical facilities. In other words, it could be the comparison of the 
extent to which the :goal and objectives of a programme have been 
achieved against the type and quantity of activities considered neces- 
sary for their accomplishment. 


Alternative participatory techniques in terms of costs must be con- 
sidered. However, the assessment of EFFICIENCY need not be based 
entirely on financial considerations. Public participation programme 
EFFICIENCY might include a ratio of costs to benefits, input to out- 


put, or external (or outside) participant perceptions to internal (official) 
perceptions. 


Lastly, EFFICIENCY may be measured by a comparison and contrast 
of programme EFFECTIVENESS to EFFORT. 


Total 


Evaluation 
Process 


Performance 
Indicators 


The Total Evaluation 


As described, the evaluation process is built upon the concepts of 
programme EFFORT, EFFECTIVENESS and EFFICIENCY. Indica- 
tors should be able to be developed to facilitate classification, 
measurement and analysis of these three concepts. 


More specifically, Performance and Process Indicators should be 
able to be developed for programme EFFORT, while Effectiveness 
and Change Indicators should be able to be developed for programme 
EFFECTIVENESS. Schematically, the total evaluation process is 


represented in Fig. II. Components in the Total Evaluation Process. 
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Fig. IZ. Components in the Total Evaluation Process 


DESCRIPTION OF PROGRAMME EFFORT INDICATORS 


Performance Indicators are descriptive devices which facilitate 
classification, measurement, and analysis of the activities of 
programme EFFORT. Examples of Performance Indicators for 
the participatory technique of a public meeting could include the 
following: 


e How many meetings were held? 


e What were their frequency and duration? 


Process 
Indicators 


Effectiveness 
Indicators 


Change 
Indicators 


e Where were the meetings held and at what time? 


e How many people attended? 


Process Indicators are descriptive devices which facilitate 
classification, measurement, and analysis of the process in- 
volved in programme EFFORT. Examples of Process Indica- 

tors might be based on the choice of the participatory techniques 
and their probability of success according to the level of experience 
or sophistication of the public. In other words, would a planning 
seminar rather than a task force have had a better chance of 
facilitating planner-public interaction with a specific target group. 


DESCRIPTION OF PROGRAMME EFFECTIVENESS INDICATORS 


Effectiveness Indicators are descriptive devices which facilitate 
classification, measurement and analysis of the degree to which 
the programme is able to facilitate public involvement. One ex- 
ample of an Effectiveness Indicator might be Accessibility. Did 
every individual directly affected have the opportunity of partici- 
pating ? 


Change Indicators are descriptive devices which facilitate classi- 
fication, measurement, and analysis of both the learning processes 
and the changed human action processes involved in programme 
EFFECTIVENESS. An example might be the increase in skills, 
public participants develop by communicating and articulating 
concerns and issues. 


ILLUSTRATION 
Having determined the objectives of the programme, the approach 
to be used, and the participants to be involved in the evaluation 
process, more attention will be turned to the three major factors 
of analysis: 

e EFFORT 


e EFFECTIVENESS 


« and EFFICIENCY 


In order to discuss more fully these three factors, we have prepared 
an illustrative public participation programme. In our example, a 
hypothetical planning consultant was hired by a regional municipality 
to conduct an 8 month study to recommend the most suitable align- 
ment for a new highway and interchange. Appendix I outlines in 
greater detail our consultant's approach to and response from the 
residents of the study area. 


Throughout the eight month study, our firm employed a wide variety 

of techniques to incorporate citizen input in the route selection pro- 
cess. These participatory techniques included the use of question- 
naires, face-to-face encounters, distribution of information, telephone 
and letter contacts, public meetings, citizen reports and submissions 
and citizen advisory committees. 


Detailed Description of the Model 


PUBLIC PARTICIPATION PROGRAMME EFFORT (INPUT) 


As previously outlined, public participation programme EFFORT 
refers to the type and quantity of programme activities considered 
necessary to achieve stated programme goals. This analysis might 
be accomplished by the use of both Performance and Process Indi- 
cators. 


Process Indicators might be utilized to determine what type of 
participatory technique(s) should be utilized. What combination of 
the following would be appropriate? 


e Distribution of e Public Meetings 
Resource Material 
e Citizen Advisory Groups 
e Use of the Mass Media 
eo Planning Seminars 
e Polls and Household 


Surveys e Task Forces 

e Communication through e Community Secretariat 
telephone conversations Models 
or letters 


e Working Committees 
e Face-to-face Encounters 


e Written Submissions e Community Boards 


PROCESS INDICATORS 


Process Indicators for programme EFFORT might be based on the 
type of public-planner interaction method or technique employed, 
utilizing Bregha's* four participatory categories of: 


* for a more detailed description of these categories, please refer 
to the Ministry of Culture and Recreation's publication, "Public 
Participation in Planning Policy and Programme" prepared by 
Francis Bregha (1973) 
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Information - feedback 
Consultation 
Joint Planning 
Delegated Authority 
These four participatory categories have been expanded to illustrate 


different participatory techniques, in Table I. Participatory Technique 
Continuum. 


Table I. Participatory Technique Continuum 


Boards 
Working Committee 
Community Secretariat 
Task Force 

Planning Seminars 
Citizens Advisory Group 
Public Meeting 

Written Submissions 
Face-to-lace Encounters 
Telephones/ letters 

Polls and Surveys 

Mass Media 

Distribution of Resource 
Material 


Working Committee 
Community Secretariat 
Task Force 

Planning Seminars 
Citizens Advisory Group 
Public Meeting 

Written Submissions 
Face-to-Face Encounters 
Telephones/ letters 

Polls and Surveys 

Mass Media 

Distribution of Resource 
Material 


Task Force 

Planning Seminars 
Citizens Advisory Group 
Public Meeting 

Written Submissions 
Face-to-Face Encounters 
Telephones/ letters 

Polls and Surveys 

Mass Media 

Distribution of Resource 
Material 


Public Meeting 

Written Submissions 
Face-to-Face Encounters 
Telephones/ letters 

Polls and Surveys 

Mass Media 

Distribution of Resource 
Mate rial 


Information Consultation Joint Delegated 
Feedback Planning Authority 


Process Indicators then could be based on the type of participatory 
technique chosen in relation to the social and political sophistication and 
nature of the community of interest to which the technique is directed. 
Probability of success of a particular participatory technique in a given 
situation should be able to be determined through the development of a 
probability matrix. 


DEGREE OF SOPHISTICATION OF 
COMMUNITY OF INTEREST 


Consultation Consultation Information 
Joint Planning Joint: Planning Feedback 
Delegated- 


PROBABILITY Authority 
OF 
Information Information- Consultation 
SUCCESS Feedback Feedback Joint Planning 
Delegated- 
Authority 


Table II. Probability Matrix 


As illustrated in Table Il, the Probability Matrix, the degree of 
sophistication should be determined by examining the community 

of interest with respect to its degree of organization, its geogra- 
phical location, its strength, its past performances, its inherent 
resources, and its access to resources. By comparing sophisti- 
cation of the group or specific public with the participatory category 
utilized, it should be possible to determine the probable success of 
the EFFORT attempted. With this in mind, the appropriate types of 
participatory techniques might be selected. 


In the case example stted on page7 you 
will recall the utilizatton of the follow- 
tng participatory techniques: questtonnatres, 
face-to-face encounters, telephone conversa- 
tions and letters, public meetings, submis- 
stons, and ctttzen advtsory committees. It 
becomes evtdent that "Consultatton" parttct- 
patory techniques were uttltzed. (Refer to 
Table I, Parttetpatory Technique Continuum, 
pg. 10). With respect to the target groups 
or communtttes of interest in the study area, 
tt becomes evident that they vary tn nature 
and sophtstteatton from low (for example, 
uninformed individuals) to high (envtron- 
mental groups). 


In thts case study, the publte groups affect- 
ed by the study range from low to high de- 
grees of sophistteatton. All four parttctpa- 
tory categortes of informatton-feedback, 


consultatton, joint planning, and delegated 
authortty should have been employed. 


No parttetpatory technique from etther the 
Jotnt Planning parttetpatory category (for 
example, working committees operating as 
partners with planners) or the Delegated 
Authority parttetpatory category (for example 
boards with delegated authority to tmplement 
thetr dectstons) were utilized. As a result, 
the effective use of the talents of such 
sophtstteated groups as an Envtronmental 
Club could have tmproved the quality of 
_public-planner information exchange. The 
probability of the public acceptance of the 
programme results could have been higher tf 
tn additton to informatton feedback and con- 
sultation, the programme administrators had 
utiltzed Jotnt Planning and Delegated Auth- 
ortty participatory techniques. 
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PERFORMANCE INDICATORS 


Performance indicators for programme EFFORT might include not only 
the number of occurrences of the previously mentioned participatory 
techniques, but also the means and extent of activity involved in each. 
Some of the indicators that may be applied to each programme option 


are described in Table II Participatory Techniques and Illustrative 
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Performance Indicators: 


Programme 
Options 


Participatory 
Technique 


Distribution 
of Resource 
Materials 


Use of Mass 
Media 


Household 
Surveys and 
Polls 


Telephone/ 
Letters 


Face-to-Face 
Encounters 


Performance Indicators 


Types of materials distributed 
Total distribution of materials 
Numbers of participants reached 


Types of media used 
Frequency of use 
Numbers of participants reached 


Methods used 
Frequency of use 
Numbers of participants reached 


Methods employed 
Frequency of use 
Numbers of participants reached 


Methods employed 
Frequency and duration of contact 
Numbers of participants reached } 


Written A. Number received | 
Submissions B. Sources 
C. Issues addressed 
A. Types of forums held 
B. Frequency and duration 
C. Location and attendance 
Citizen A. Number established 
Advisory B. Frequency and duration 
Groups C. Numbers of participants involved 
Planning A. Number held 
Seminars B. Frequency and duration 
C. Numbers of participants involved 


Community 
Secretariats 


Working 
Committees 


Board 


Task 
Forces 


Numbers established 
Frequency and duration of meetings 


Number of participants involved 


Number established 
Location 
Resources available 


Number and type established 
Frequency and duration 
Number of participants involved 


Number and type established 
Frequency and duration of meeting 
Number of participants involved 


Table Il. Participatory Techniques and Illustrative 
Performance Indicators 


In our study, performance tndtcators 
could tnelude such detatls as follows: 


A) questtonnatres 


- 15,000 dtstrtbuted 

1,000 returned 

6.6 return rate 

analysts by area 

east 8% 

north 25% 

west 04% 

south 10% 

unknoum 3% 

analysts by afftltatton 

agriculture 20% 

envtronmental 5% 

communtty group 14% 

government o% 

bustness o% 

tndivtduats 51% 

analysts of responses - 

tn favour of a northern 
route corridor 68% 

tn favour of a western 
route 352% 

ete. 


B) publte meetings 


- number of meetings 23 
- average attendance 101 


Purpose Date Place Attendance 


to present .26,1976 Untverstty City 
environmental .27,1976 Riverview 

maps and obtatn 28,1976 Bushtown 

Local comments, .29,1976 Lakestoun 
corrections and .30,1976 Waterville 
additions 


LO 
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C) ettizens advisory committees 


- number established - 8 
- number of parttetpants - 16 


- tssues addressed 
a) ecological tmpact 
b) community tmpact 
ete. 


PUBLIC PARTICIPATION PROGRAMME EFFECTIVENESS (OUTPUT) 


Once it has been determined which combination of the participatory 
techniques are to be used with their respective Performance and Pro- 
cess Indicators, it is possible to develop measures of effectiveness. 
As outlined, EFFECTIVENESS refers to the assessment of the extent 
to which goals and objectives of the public participation programme 
have been met. In order to accomplish this analysis, objectives must 
be clear, specific, and sufficiently concrete in order that EFFORT 
may be related to them. EFFECTIVENESS of the public participation 
programme ought to be able to be measured by the use of both effect- 
iveness indicators and change indicators. 


EFFECTIVENESS INDICATORS 


Effectiveness Indicators which facilitate the classification, measure- 
ment, and analysis of the degree to which the programme is able to 
facilitate public involvement, might include measures such as: 


e Accessibility 
Every individual directly affected has the opportunity of 


participating. Obviously, if effort has not been made 
either in choices of adequate means or sufficient resources 
used to enable the affected public to become involved, there 
will not be adequate accessibility by the public to planners, 
information, and decision-making. 


r Credibility 
Planners are consistent in their dealings with the public. 
If more than one individual is responsible for the programme, 
then a continuous review is necessary to maintain a completely 
coherent programme. Obviously, if the programme does not 
have credibility with the public, it makes little difference how 
much effort went into establishing it. 


Comprehensiveness 


The nature of the programme as a whole must be communicated 
to those whom it will affect. Technical jargon must be replaced 
by everyday language. Again, effort could be wasted in establish- 
ing a programme in which the public would have real difficulties 
participating if residents couldn't see where or how they fit into 
the process, and the types of contributions they could make. 


Impartiality 


No individual or group is allowed to dominate the participation 
process. Every affected person has an equal opportunity to 


participate. Care should be taken that the programme 
options insure this openness in determining programme 
effort. 


Flexibility 

The programme is flexible to accommodate the unexpected. 
This is an important aspect of any public participation pro- 
gramme. When planners invite the public to participate, it 
should be expected that changes may have to be made in 
programme schedule and effort should be made to accom- 
modate this. 


Acc ountability 


Citizens and planners equally are accountable for their 
decisions. Again, an important aspect from the point of 
view of increased costs due to decisions jointly made. 


Integration 
The public participation programme is fully integrated 


with and complementary to the overall planning process. 
It should not be a token effort "tacked on" to the planning 
programme and added to the responsibility of the planning 
staff. 


Maximum Contact 

Direct contact with the public is maximized through both 
formal and informal channels. Obviously, programme 
EFFORT should reflect this. 


Training and Experience 


There is adequate staff, in terms of number, qualifications 
and training available for the Public Participation Programme. 
The problems here are self evident if a successful programme 
is expected. 


Structure 

The organization structure of the programme is conducive to 
its goals and objectives; a key requirement if the programme 
is to be evaluated. 
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Utilizing total programme EFFORT, assessment of EFFECTIVENESS, 


based on effectiveness indicators could be as follows: 


Total 
Programme 
EFFORT 


- Distribution of 
Resource Material 
- Mass Media 


- Household Surveys 
and polls 


- Telephone/ Letters 


-—- Face-to-Face 


Encounters 
- Written Submissions 
- Public Meetings 


- Citizen Advisory 
Groups 


- Planning Seminars 

- Task Forces 

- Community Secretariat 
- Working Committees 


- Boards 


EFFECTIVENESS 
Indicators 


Accessibility 
Credibility 
Comprehensibility 
Impartiality 
Flexibility 
Accountability 
Integration 
Maximum Contact 


Training and 
Experience 


Structure 


Table IV. [Illustrative EFFECTIVENESS Indicators 


For example 


Examining the Total Programme EFFORT of 
the public parttctpatton component of 

the case example, assessment of programme 
EFFECTIVENESS by means of EFFECTIVENESS 
Indtcators could consider such questions: 


Aecesstbiltty - Did every indivtdual 
affected have the opportuntty 
of parttictpating? Did the pub- 
lie have easy access to tnfor- 
mation and to planners? Might 
planners have uttltzed store- 
front operations or local 
engineering and works depart- 
ment offtctals and offices to 
faetlttate a greater planner 
publte exchange? 


Credtbility - Were planners conststent 
tn thetr dealings with the 
public? Did local citizens 
feel that the study was cre- 
dtble or that perhaps dectstons 
had already been made by plan- 
ners? Was there adequate 
planner-publte tnterchange tn 
order to develop a proper rap- 
port? 


Comprehenstbiltty - Was the programme 
understood by those affected? 
Were discusston papers, back- 
ground papers and progress 
reports stmple, logteal and 
readable by the affected 
publte? Were they able to 
develop a well rounded under- 
standing of the study purpose, 
constraints, and options? 


Imparttaltty - Did tndividual cttizens 
have as equal an opportuntty 
as organtzed groups to present 
thetr opintons? 


Contaet - Was publtc-planner 
contact and interactton maxt- 
mized through formal and tn- 
formal encounters? Was the 
eight month study pertod long 
enough for an effective,public 
parttetpatton programme? 


ay) 


CHANGE INDICATORS 


In addition to measuring the degree to which the programme is able to 
facilitate public involvement, it is necessary to measure both the re- 
lated learning process and changes within the process itself. 


Change Indicators which may facilitate this type of programme 
EFFECTIVENESS could include the following: 


e The frequency of contact of groups and individuals with 
elected representatives and officials. e.g. The number 
of meetings, telephone calls, letters, delegations, etc. 


e Examples of a change in the nature of the plan; in other 
words, indication that affected and interested citizens 
have had an effect on the outcome. e.g. Changes inplan 
design implementation, etc. 


e Frequency of examples indicating a new change in self- 
perception. e. g. New leaders in groups, previously 
inarticulate individuals speaking out, etc. 


e Frequency of personal initiatives to further self-devel- 
opment. e.g. Attendance of seminar courses, reading 
of resource material, etc. 


e Increased skill in communicating and articulating con- 
cerns and tackling new issues. e.g. Individual applica- 
tions of previous learning to new situations, individual's 
ability to change from critic to constructive participant 
in seeking alternatives, etc. 


e Increase in involvement in community activity. e.g. 
Formation of new groups, new memberships on com- 
munity committees, etc. 


e Frequency of travel and contacts outside area of interest. 
e.g. Individuals acting as resource people to other com- 
munities on similar issues, attending conferences etc. 


e Increase in utilization of community resources. e.g. 
Libraries, community agencies, indigenous expertise, 
etc. 


e Indications of constructive interaction e.g. Joint co-op- 
eration with community agencies on issues, umbrella 
groups, coalitions, etc. 


e Frequency of individuals accepting personal responsibility 
for problem resolution. e.g. Memberships on committees, 
briefs prepared, surveys, petitions, etc. 


Regarding total Programme EFFORT; some change indicators could be as follows: 


Table V. Illustrative Change Indicators 


Total 


Programme OEE 


Effort Indicators 


Frequency of 
Contact 


Examples of 
Changes on 
Outcome 


Changes in Self 
Perception 


Initiatives in 
Self Development 


Communication 
Skills 


Outside contacts 


Increased 
Involvement 


Increased Use of 
Resources 


Constructive 
Interacting 


Acceptance of 
Personal Res- 
ponsibility 
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In the case study, analysts of change 
tndteators might tneclude: 


- The frequency of contact of groups 
and the publte with programme plan- 
ners and administrators. t.e. project 
staff met wtth X number of groups Y 
times over a Z length of time. 


Examples of changed quality of situa- 


tion or indicattons. that tnterested 
etttzen tnvolvement had an effect on 

the final study recommendations. 1.é. 
the final route recommendations differed 
from original recommendattons. 


Increased number of the publie tnteres- 
ted tn the quality of thetr environment 
t.e. ereation of new ettizen organtza- 
ttons to handle new tssues. 


Finally, the model includes an analysis of programme EFFICIENCY. 


PUBLIC PARTICIPATION PROGRAMME EFFICIENCY 


A vital component of any evaluation is an analysis of programme 
EFFICIENCY. Unfortunately, this segment of the evaluation is often 
never undertaken. However, this analysis is necessary in order that 
lessons can be learned and that in future more effective alternate 
resources and participatory techniques may be utilized. EFFICIENCY 
may be considered as a ratio between the performance criteria applied 
to that programme based on initial objectives and assumptions. This is 
expressed in Table VI. Programme Efficiency Chart, which follows. 


Performance Performance Performance 
Assumptions Criteria Results 


Objective 


- that citizens' - that citizens Effort Efficiency 
opinions and believe the and 
concerns are issue to be of Effectiveness 
incorporated enough import- 
into the tech- ance to become 
nical studies involved 
which form 
the basis for - that the 
recommend- technical con- 
ations siderations 
are compre- 
hensible to the 
average citizen 


Table VI. Programme Efficiency Chart 


The essence of programme EFFICIENCY is highlighted in this question. 
Could the same or better performance results be achieved by either 
reducing the amount of public participation programme EFFORT or by 
choosing other kinds of public participation programme EFFORT which 
are less costly in achieving the same objective? 


Without hampering the success of the programme 
could the costs of the project study have been 
reduced by the uttlizatton of other equally 
effective participatory techniques? Could. 
fewer public meetings and more working com- 


mitttees have produced more effective public- 
planner interaction? Did the etght month 
study time pertod limit the use of certain 
combinations of participatory techniques? 
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The Outcome 


Instead of providing a check list for the evaluation of public participa- 
tion programmes, the purpose of this paper is to stimulate a thought 
process amongst planners who would like to or who may be embarking 
upon an evaluation of local public participation programmes. 


In order to further refine and develop public participation programmes, 
practitioners must recognize the necessity to empirically document and 
analyze programme processes. Evaluation of public participation pro- 
grammes for policy development is necessary and manageable. By 

means of utilizing the interrelated concepts of EFFORT and EFFECTIVE- 
NESS, the final factor EFFICIENCY is determined by means of devising 

a ratio of public participation EFFORT to EFFECTIVENESS. The use of 
these three concepts will result in an inclusive and useful evaluative tool 
from which we can all learn. 


Glossary 


Change Indicators 


Change indicators are descriptive devices which facilitate classifi- 
cation measurement, and analysis of both the learning processes 
and the change of human action processes, involved in programme 
EFFECTIVENESS. 


Citizens' Advisory Group 
The Citizens' Advisory Group is a group of representive citizens 


who are either elected by the community or appointed by planning 
officials to represent local interests by consulting and advising 
with planners in the planning process. 


Community Secretariat 


A Community Secretariat is a non-partisan community resource 
Service in which all community groups have equal access to its 
resources of typing and duplication, central information, planning 
consultation, volunteer support to groups, leadership develop- 
ment and intergroup liaison. 


Effectiveness Indicators 
Effectiveness Indicators are descriptive devices which facilitate 
classification measurement and analysis of the degree to which 
the programme is able to facilitate public involvement, and are 
used in programme EFFECTIVENESS. 


Evaluation ! 
Evaluation is the systematic gathering of data to provide informa- 
tion about the achievement of programme requisites and goals 
relative to efforts, effectiveness and efficiency. The facts of 
evaluation may be obtained through a variety of relatively sys- 
tematic techniques, and they are incorporated into some desig- 
nated system of values for making decisions about social problems. 


Indicators 
Indicators are descriptive devices derived from social science 
research which facilitate classification, measurement and analysis 
of behaviour, results and processes. 


Participatory Techniques 


Participatory techniques are methods of involving the public in 
planning processes through constructive interaction with planning 
officials. A continuum of participatory techniques might include 
distribution of resource material, mass media techniques, surveys 
and polls, telephoning/letters, face-to-face encounters, written 
submissions, public meetings, citizens' advisory groups, plan- 
ning seminars, task forces, community secretariats, and 

boards with delegated authority. 


Performance Indicators 
Performance Indicators are descriptive devices which facilitate 
classification measurement, and analysis of the activities of 
programme EFFORT. 


Process Indicators 
Process Indicators are descriptive devices which facilitate classi- 
fication, measurement, and analysis of the process involved in 
programme EFFORT. 


Programme EFFECTIVENESS 


This concept refers to the extent to which the goals and objectives 
of the programme have been achieved. 


Programme EFFICIENCY 
This concept refers to the relationship between the EFFECTIVE- 


NESS and EFFORT, in terms of such costs as money, time, per- 
sonnel, convenience, and physical facilities. 


Programme EFFORT 


This concept can be viewed as the type and quantity of activities 
necessary for the accomplishment of programme goals - regardless 
of output. 


Public Participation Programme 


Usually initiated by planners, a public participation programme is 
a process of interaction amongst planners and the public, and is 
established for the purpose of involving specific publics in decision 
making processes, by a variety of participatory techniques. 


Appendix 


First Questionnaire 


The first questionnaire, included in a brochure describing the study, 
was distributed to those attending three series of regional meetings, 
to those on the mailing list, and, in bulk, to municipalities, local 
groups and associations. Respondents were asked to rank the signi- 
ficance of various factors in selecting their preferred alternative. 


There is no way to estimate accurately how many were, in fact, passed 
on to individuals from the approximately 15,000 distributed in this 
manner. Approximately 1,000 of those distributed were eventually re- 
turned. A computer analysis of 906 of these gave a fairly clear indica- 
tion of the relative weights which respondents attached to various en- 
vironmental features. 


Local Date Collection 


At a municipal meeting, those attending stressed the importance of 
obtaining as much local input to the inventory of data as possible. With 
this in mind, copies of the base map of 1:200 scale, along with loose 
leaf note books containing instructions on how to mark the maps and 
giving examples of information that would be useful were prepared. A 
description of other inventory maps and their contents was also in- 
cluded in this kit. 


At the first series of regional meetings, the purpose of these data books 
and maps was explained, and their importance to the study was stressed. 
A request was made for volunteers to co-ordinate and publicize the 
gathering of information in their local areas. In all, twelve kits were 
distributed throughout the study area. It was the responsibility of some 
volunteers to make these kits known and accessible to their respective 
communities. Considerable time was spent in the small discussion 
groups outlining suggestions as to where the maps should be placed. 

The maps and comments were returned three weeks after distribution, 
when they were mapped for display at the Series II meetings. 


Fourteen of these volunteers were private citizens while five were 
elected representatives or appointed officials. 
Submissions 


During the 1976 summer hearings, 134 submissions were received, 5 
from municipal departments, 30 from municipalities, 18 from groups 
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Submissions (cont'd) 


and associations and 60 from property owners. Each submission was 
carefully reviewed and wherever possible, local details on land fea- 
tures were mapped in the local data collection inventory. In general, 
those who made submissions fell into three categories: municipal 
departments, those groups with a general environmental interest, and 
property Owners. 


Corre spondence 


During the course of the study up to the third series of regional meetings, 
letters were received from individuals and groups throughout the study 
area. In addition to requests for additional information, many of the 
letters included specific local data which was recorded and mapped. In 
general, comments received by mail reiterated those concerns most 
often stated at meetings. Following the second series of regional meet- 
ings, letters and numerous phone calls were received giving detailed 
information to be added to the inventory maps. As with more formal 
submissions the correspondence we received from individuals expressed 
a deep concern for the direction and management of "growth" in the study 
area, and the processes by which decisions on growth are made. 


Queen's Park Meetings 


Meetings at Queen's Park were held on ten occasions throughout the 
course of the study and were consistently attended by a number of re- 
gular participants who had been involved in the previous hearings. 


They included representatives of provincial and municipal governments, 
citizens' and conservation groups, as well as some individuals who 
were directly concerned. 


For the most part, these participants were well informed about the 
background of the study and its purpose, and were aware of the broad 
range of problems involved. Attendance at these meetings usually ranged 
between 60 and 90 persons. 


Regional Meetings 


The three series of regional meetings were scheduled to coincide with 
the three stages of the study. The Series I meetings introduced the 
reasons for the study and explained the methodology involved. The en- 
vironmental inventory maps were presented at Series I; and the pro- 


Regional Meetings (cont'd) 


posed alternative systems and routes formed the focal point of Series 
Ill. The purpose of all meetings was not only to report on progress of 
the study but also to obtain as much direct public input as possible. 


Background information, and descriptions of the work done to that point, 
were distributed at each series of meetings. 


In each series, a report of the study's progress to date was included for 
those new to the study. An accompanying slide presentation was followed 
by a coffee break and questions. Meetings were taped, and all questions 

and comments reviewed. 


About 400 persons took part in four regional meetings of Series I; 700 
turned out to the four meetings of Series IJ; while the nine meetings of 
Series II attracted a total of 1,600. An additional 200 individuals atten- 
ded the five Queen's Park meetings of Series II. 


At the regional meetings, participants consisted primarily of those 
living close to the place where the meetings was held. Usually the main 
concern was with the potential impact of the project on their own pro- 
perty or community. 


Publicity 


The plan to achieve the fullest possible coverage for the study included 
regular mailings to every newspaper, radio and television station in the 
study area. These dispatches included press releases and radio/TV spot 
announcements as well as background material. Generally the publicity 
campaign aimed toward a maximum public awareness of the meetings and 


toward a continued reporting of developments. During the first and second 


set of regional meetings, members of the press attending were individually 


approached and personal contact maintained throughout the study where- 
ever possible. A news clipping service was retained to monitor coverage 


received from both local and regional press. Advertisements were placed 


in every paper in the region prior to each series of regional meetings. 


At each stage of the study, explanatory background information and pro- 
gress reports were provided, and advance notices of all meetings held 
were distributed to all those on the mailing list. To ensure delivery, 
letters introducing the study, notices of all meetings and information 
regarding the proposed alternative routes was sent to the municipalities 
by registered mail. 
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